Cardiac emergencies are the leading cause of mortality worldwide. In a cardiac emergency, the two pillars that facilitate the right diagnosis and treatment are; the time and accuracy of every verbal or diagnostic information. Both of these have a critical effect on the outcomes of the cardiac patient.
In terms of diagnosis, Electrocardiograms (ECG) are one of the most commonly used tests in the medical industry to diagnose and monitor heart conditions. They are used to evaluate the electrical activity of the heart, detect abnormalities and identify underlying heart diseases.
Most studies on machine learning classification of electrocardiogram (ECG) diagnoses focus on processing raw signal data rather than ECG images.
This presents a challenge for models in many areas of clinical practice where ECGs are printed on paper or only digital images are accessible.
In our graduation project, our aim is always to develop models that are practical to use in the real world processes and production environments of cardiovascular health domain.
So as part of this vision, we aim to develop and evaluate the accuracy of models of images based machine learning algorithms on 12-lead ECG diagnosis.
An electrocardiogram (ECG or EKG) records the electrical signal from the heart to check for different heart conditions. Electrodes are placed on the chest to record the heart's electrical signals, which cause the heart to beat. The signals are shown as waves on an attached computer monitor or printer.
ECG signals are represented via a graph of the electrical activity of the heart. It is made up of a series of waves, called P, Q, R, S, and T waves. These waves represent the different stages of the heart's electrical cycle.
P wave represents the electrical signal that spreads through the atria, the two upper chambers of the heart. The atria contract, pushing blood into the ventricles, the two lower chambers of the heart.
QRS complex represents the electrical signal that spreads through the ventricles, causing them to contract and pump blood out to the body.
T wave represents the electrical signal that spreads through the ventricles as they relax.
PQRS interval is the time between the beginning of the P wave and the end of the S wave. This interval represents the time that it takes for the electrical signal to travel from the atria to the ventricles and cause the ventricles to contract.
QT interval is the time between the beginning of the Q wave and the end of the T wave. This interval represents the time that it takes for the electrical signal to travel through the ventricles and cause them to contract and relax.
The PQRST waves and intervals on an ECG signal can be used to diagnose a variety of heart conditions, including arrhythmias, conduction defects, and myocardial infarction.
Compared to a single-lead or 6-lead ECG, a 12-lead ECG provides more detailed information about the heart’s electrical activity, making it a superior tool for diagnosing and monitoring a range of cardiac conditions.
Each of the 12 EKG leads represent a different direction of cardiac activation in 3-D space. The standard EKG leads are denoted as lead I, II, III, aVF, aVR, aVL, V1, V2, V3, V4, V5, V6. Leads I, II, III, aVR, aVL, aVF are denoted the limb leads while the V1, V2, V3, V4, V5, and V6 are precordial leads.
ECG images dataset of Cardiac Patients created under the auspices of Ch. Pervaiz Elahi Institute of Cardiology Multan, Pakistan that aims to help the scientific community for conducting the research for Cardiovascular diseases.
The dataset contains ECG images of patients with myocardial infarction, abnormal heartbeat, history of MI, and normal ECGs.
The link of the dataset is provided her https://data.mendeley.com/datasets/gwbz3fsgp8/2
1. Myocardial_Infarction_Patients/
2. Abnormal_Heartbeat_Patients/
3. History_of_MI_Patients/
4. Normal_Person_ECG_Images/
Understanding the medical background of the four categories in the dataset is paramount as it forms the foundation for addressing the nuanced challenges and the cardiac domain. Before delving into coding and machine learning algorithms, let's have a comprehension of the medical context.
A myocardial infarction (MI), commonly known as a heart attack, is an extremely dangerous condition that occurs when blood flow decreases or stops in one of the coronary arteries of the heart, causing infarction (tissue death) to the heart muscle.
Most MIs occur due to coronary artery disease ,which is considered a crucial concern that we have also worked on through the Framingham heart study dataset through our graduation project,.
Risk factors of MI include family history of heart disease, high blood pressure, smoking, diabetes, lack of exercise, obesity, high blood cholesterol.
ECG is considered one of the leading helpful tests to help with diagnosis of MI.
Note :
The key difference between the two categories of "ECG Images of Myocardial Infarction Patients" and "ECG Images of Patients with History of MI" in the dataset lies in the timing and context of the collected electrocardiogram (ECG) images:
1. ECG Images of Myocardial Infarction Patients:
2. ECG Images of Patients with History of MI:
Abnormal heartbeat or refered as heart arrhythmia is an irregular heartbeat. A heart arrhythmia occurs when the electrical signals that tell the heart to beat don't work properly. The heart may beat too fast or too slow. Or the pattern of the heartbeat may be inconsistent.
Some heart arrhythmias are harmless. Others may cause life-threatening symptoms.
Risk factors include coronary artery disease, High blood pressure, smoking and other medical problems
Possible complications of heart arrhythmias may be dangerous and may reach to include:
Normal individuals without a history of cardiac issues are considered as a baseline or reference. Normal ECG patterns show regular electrical activity in the heart, providing a standard for comparison. This category serves as a control group in the dataset
import os
import cv2
import matplotlib.pyplot as plt
from matplotlib import gridspec
import seaborn as sns
import numpy as np
import pandas as pd
import sklearn
import re
import warnings
warnings.filterwarnings("ignore", category=UserWarning)
from skimage.segmentation import slic
from skimage.color import label2rgb
from skimage.filters import threshold_otsu,gaussian
from skimage import measure
from sklearn.preprocessing import MinMaxScaler
from skimage.io import imread
from skimage import color
from skimage.transform import resize
# Path to the main folder
main_folder = r"C:\Users\user\Desktop\Menedelay ECG Data"
# Subfolders for each category
categories = [
"Myocardial Infarction Patients",
"Abnormal Heartbeat Patients",
"History of MI Patients",
"Normal Person ECG"
]
# Let's load and visualize a few sample images
for category in categories:
category_folder = os.path.join(main_folder, category)
image_files = os.listdir(category_folder)
sample_image_path = os.path.join(category_folder, image_files[0])
# Read the image using OpenCV
img = cv2.imread(sample_image_path, cv2.IMREAD_GRAYSCALE)
# Visualize the images
plt.imshow(img, cmap='gray')
plt.title(f"Sample Image from {category}")
plt.show()
Let's provide more context on the ECG images in our data. The ECG images in the dataset are consistent with the standard ECG calibration :
Each large square on the ECG image represents 0.2 seconds, and each small square represents 0.04 seconds. The amplitude of the waveforms is measured in millivolts (mV), and a signal with an amplitude of 1 mV moves the recording stylus vertically 1 cm.
Therefore, we can interpret the ECG images in the dataset as follows:
Each large square on the ECG image represents 0.2 seconds of the patient's heartbeat.
Each small square on the ECG image represents 0.04 seconds of the patient's heartbeat.
The amplitude of the waveforms on the ECG image is measured in millivolts (mV).
A signal with an amplitude of 1 mV moves the recording stylus vertically 1 cm.
This information in the medical domain can be used to measure various parameters of the patient's heartbeat, such as the heart rate, the duration of the P-wave, QRS complex, and T-wave, and the amplitude of the R-wave.
# Let's visualize the distribution of the 4 classes of our data
class_counts = []
for category in categories:
category_folder = os.path.join(main_folder, category)
image_files = os.listdir(category_folder)
class_counts.append(len(image_files))
with plt.style.context('fivethirtyeight'):
custom_palette = ['#fe346e', '#512b58', '#ffb037', '#4da6ff']
sns.barplot(x=categories, y=class_counts, palette=custom_palette)
plt.xticks(rotation=90)
plt.title("Class Distribution")
plt.show()
ECG signals are recorded from 12 leads, and analyzing them separately provides valuable information about different aspects of the heart's function. Let's build a function to seperately view each lead so we have a closer look into the ECG images
# Define lead coordinates
lead_coordinates_1_to_12 = [
(300, 600, 150, 643),
(600, 900, 150, 643),
(900, 1200, 150, 643),
(300, 600, 646, 1135),
(600, 900, 646, 1135),
(900, 1200, 646, 1135),
(300, 600, 1140, 1625),
(600, 900, 1140, 1625),
(900, 1200, 1140, 1625),
(300, 600, 1630, 2125),
(600, 900, 1630, 2125),
(900, 1200, 1630, 2125),
]
# Define lead names
lead_names_1_to_12 = [
"Lead I", "Lead II", "Lead III",
"Lead aVR", "Lead aVL", "Lead aVF",
"Lead V1", "Lead V2", "Lead V3",
"Lead V4", "Lead V5", "Lead V6"
]
# Define coordinates for Lead 2 prolonged
lead_2_long_coordinates = (1250, 1480, 150, 2125)
# Function to extract a lead from an image based on coordinates
def extract_lead(image, coordinates):
lead = image[coordinates[0]:coordinates[1], coordinates[2]:coordinates[3]]
return lead
# Function to visualize leads 1-12
def visualize_leads_1_to_12(image_path):
# Load the ECG image
image = cv2.imread(image_path, cv2.IMREAD_GRAYSCALE)
# Divide the ECG into leads
leads = [extract_lead(image, coord) for coord in lead_coordinates_1_to_12]
# Plotting lead 1-12
fig, axs = plt.subplots(4, 3, figsize=(20, 20))
for i, (lead, ax, name) in enumerate(zip(leads[:12], axs.flatten(), lead_names_1_to_12), 1):
ax.imshow(lead, cmap='gray')
ax.set_title(name)
plt.show()
# Function to visualize Lead 2 prolonged
def visualize_lead_2_prolonged(image_path):
# Load the ECG image
image = cv2.imread(image_path, cv2.IMREAD_GRAYSCALE)
# Extract Lead 2 prolonged from the image
lead_2_long = image[lead_2_long_coordinates[0]:lead_2_long_coordinates[1], lead_2_long_coordinates[2]:lead_2_long_coordinates[3]]
# Plotting Lead 2 prolonged
fig, ax = plt.subplots(figsize=(20, 20))
ax.imshow(lead_2_long, cmap='gray')
ax.set_title("Lead 2 prolonged")
plt.show()
we can use these two functions we have just built to have a closer analytical look into any ECG image of our dataset. let's have a look into a sample of each class of our dataset
# Let's visualize a sample of each class
# Normal ECG leads
image_path_normal = r"C:\Users\user\Desktop\Menedelay ECG Data\Normal Person ECG\Normal(1).jpg"
visualize_leads_1_to_12(image_path_normal)
visualize_lead_2_prolonged(image_path_normal)
We can observe the 12 leads as we have discussed before. In addition to this, to assess the cardiac rhythm accurately, a prolonged recording from one lead is used to provide a rhythm strip.
Lead II is the most common, most popular, and generally the best view which usually gives a good view of the P wave and so commonly used to record the rhythm strip
By setting the ECG monitor to Lead II as we can see through the ECG images we have just plotted, we are essentially viewing the impulse as it travels from the right atria toward the left ventricle; hence, Lead II is the “best seat in the house” for viewing the wavefront. On a telemetry unit, we will see most, if not all monitors are set to Lead II for this reason.
# MI ECG leads
image_path_MI = r"C:\Users\user\Desktop\Menedelay ECG Data\Myocardial Infarction Patients\MI(6).jpg"
visualize_leads_1_to_12(image_path_MI)
visualize_lead_2_prolonged(image_path_MI)
# History of MI ECG leads
image_path_MI_history = r"C:\Users\user\Desktop\Menedelay ECG Data\History of MI Patients\PMI(20).jpg"
visualize_leads_1_to_12(image_path_MI_history)
visualize_lead_2_prolonged(image_path_MI_history)
# Abnormal ECG Leads
image_path_abnormal = r"C:\Users\user\Desktop\Menedelay ECG Data\Abnormal Heartbeat Patients\HB(12).jpg"
visualize_leads_1_to_12(image_path_abnormal)
visualize_lead_2_prolonged(image_path_abnormal)
Until now we have discussed the medical background of our data and our goals of developing a model to detect abnormalities through the 12 lead ECG images.
We then have initialized our coding with some data loading and exploring to get an overview of the dataset.
Here we reach to the core of our 1st notebook : ECG Images Processing
Later in the further steps , we will automate the entire image processing process in addition to data storage to all the set of ECG images in our dataset.
But for now, let's start walking through and processing only a MI Patient ECG sample. This will allow to understand and verify the steps we will implement.
Let's first load the sample image, then we will dive into the steps of the images processing techniques we will use
fig, ax = plt.subplots()
fig.set_size_inches(20, 20)
image = plt.imread(r"C:\Users\user\Desktop\Menedelay ECG Data\Myocardial Infarction Patients\MI(14).jpg")
ax.imshow(image)
plt.show()
# Let's first have a quick look on the image
# Let's review our leads coordinates
Lead_1 = image[300:600, 150:643]
Lead_2 = image[300:600, 646:1135]
Lead_3 = image[300:600, 1140:1625]
Lead_4 = image[300:600, 1630:2125]
Lead_5 = image[600:900, 150:643]
Lead_6 = image[600:900, 646:1135]
Lead_7 = image[600:900, 1140:1625]
Lead_8 = image[600:900, 1630:2125]
Lead_9 = image[900:1200, 150:643]
Lead_10 = image[900:1200, 646:1135]
Lead_11 = image[900:1200, 1140:1625]
Lead_12 = image[900:1200, 1630:2125]
Lead_2_prolonged = image[1250:1480, 150:2125]
Leads=[Lead_1,Lead_2,Lead_3,Lead_4,Lead_5,Lead_6,Lead_7,Lead_8,Lead_9,Lead_10,Lead_11,Lead_12,Lead_2_prolonged]
Each lead undergoes preprocessing to enhance relevant features for subsequent analysis.The key preprocessing steps we will perform include:
Binarization will classify each pixel value as either black (0 represents the background) or white (1 represents the foreground) based on its intensity level or gray-level compared to the threshold value.
This will make it easier to separate the foreground from the background and allow better features extraction from images.
There are various image thresholding techniques used. Through this notebook, we will use Otsu's Method for automatically determining the optimal threshold value in image segmentation.
It calculates the threshold by maximizing the between-class variance of pixel value, which effectively separates foreground and background regions.
# image transformation
fig , ax = plt.subplots(4,3)
fig.set_size_inches(20, 20)
#looping through image list containg all the 12 leads
for i,j in enumerate(Leads[:len(Leads)-1]):
# Convert image to grayscale_image
grayscale_image = color.rgb2gray(j)
# Apply Gaussian filtering to smooth the image
blurred_image = gaussian(grayscale_image, sigma=0.7)
# Apply Otsu's thresholding to distinguish foreground and background
global_threshold = threshold_otsu(blurred_image)
binary_image = blurred_image < global_threshold
# Resize the binary image
binary_image = resize(binary_image, (300, 450))
ax[i // 3, i % 3].imshow(binary_image, cmap="gray")
ax[i // 3, i % 3].axis('off')
ax[i // 3, i % 3].set_title(f'Processed Lead {i + 1}')
# plot the image
plt.show()
# Let's perform the same process for lead 2 prolonged
fig , ax = plt.subplots()
fig.set_size_inches(20, 20)
grayscale_image_2 = color.rgb2gray(Leads[-1])
blurred_image_2 = gaussian(grayscale_image_2, sigma=0.7)
global_threshold_2 = threshold_otsu(blurred_image_2)
binary_image_2 = blurred_image_2 < global_threshold_2
ax.imshow(binary_image_2,cmap='gray')
ax.set_title("Processed Lead 2 prolonged")
Text(0.5, 1.0, 'Processed Lead 2 prolonged')
Good so far , after we have applied filtering and thresholding, The next step will be to convert the transformed processed leads to signals.
Before diving into it, let's quickly define a custom function for plots formatting for the next steps
def apply_custom_style(ax):
# Set background color
ax.patch.set_facecolor('#f6f5f5')
# Set axis color
ax.set_facecolor('#f6f5f5')
# Remove spines
ax.spines['top'].set_visible(False)
ax.spines['right'].set_visible(False)
# Remove ticks
ax.tick_params(axis='both', which='both', length=0)
# Set tick labels font
ax.set_xticklabels(ax.get_xticklabels(), **{'font': 'serif'})
ax.set_yticklabels(ax.get_xticklabels(), **{'font': 'serif'})
This process will involve a number of steps we will work through. through each step we will usually save the data to csv files for inspection and easy access and integration into machine learning workflows in the further steps.
In this step we will identify and extract the contour of the signal within each lead.
Contours represent the shape of the ECG signal, and extracting them isolates the relevant information for classification and allows us to extract the signal from image.
# Finding contours
contours = measure.find_contours(binary_image, 0.9)
# Plotting the image with contours found
fig, ax = plt.subplots()
# Inverting the y-axis for proper visualization
plt.gca().invert_yaxis()
# Sorting contours by shape and selecting the largest one
contours_shape = sorted([x.shape for x in contours])[::-1][0:1]
# Printing the contours shape for reference
print(f'counter found : {contours_shape}')
# Plotting the selected contour
for contour in contours:
if contour.shape in contours_shape:
resized_contour = resize(contour, (255, 2))
ax.plot(contour[:, 1], contour[:, 0],color='#5c5a5a', linewidth=0.6)
# Setting axis properties
ax.axis('image')
ax.set_title("Sample processed lead V6 image",{'font': 'serif'})
apply_custom_style(ax)
plt.show()
counter found : [(2564, 2)]
# Saving the contour data to a CSV file
df_2d = pd.DataFrame(resized_contour, columns=['x', 'y'])
df_2d.to_csv('lead_V6_2d_sample.csv', index=False)
# Viewing the CSV file to verify
lead_V6_2d_sample = pd.read_csv('lead_V6_2d_sample.csv')
lead_V6_2d_sample
| x | y | |
|---|---|---|
| 0 | 131.166171 | 3.712490 |
| 1 | 136.713919 | 7.229888 |
| 2 | 142.208268 | 10.527797 |
| 3 | 140.767426 | 12.583241 |
| 4 | 143.389727 | 13.402193 |
| ... | ... | ... |
| 250 | 166.326372 | 14.143001 |
| 251 | 159.087929 | 11.476252 |
| 252 | 156.503094 | 7.538017 |
| 253 | 151.548330 | 4.697680 |
| 254 | 144.791697 | 2.131309 |
255 rows × 2 columns
In this step we will normalize the signal data to a consistent scale with MinMax Scaler to ensure that features are on similar scale, which is crucial for many machine learning algorithms that we will apply to perform optimally.
# Scaling the contour data
scaler = MinMaxScaler()
fit_transformed_data = scaler.fit_transform(df_2d)
normalized_scaled_df = pd.DataFrame(fit_transformed_data, columns=['x', 'y'])
normalized_scaled_df
| x | y | |
|---|---|---|
| 0 | 0.482524 | 0.003585 |
| 1 | 0.527031 | 0.011560 |
| 2 | 0.571109 | 0.019038 |
| 3 | 0.559550 | 0.023698 |
| 4 | 0.580587 | 0.025555 |
| ... | ... | ... |
| 250 | 0.764597 | 0.027234 |
| 251 | 0.706526 | 0.021188 |
| 252 | 0.685789 | 0.012259 |
| 253 | 0.646040 | 0.005819 |
| 254 | 0.591835 | 0.000000 |
255 rows × 2 columns
# Plotting the scaled data to test signal shape
df_2d = pd.DataFrame(normalized_scaled_df, columns = ['x','y'])
fig_scaled, ax_scaled = plt.subplots()
plt.gca().invert_yaxis()
ax_scaled.plot(normalized_scaled_df['y'], normalized_scaled_df['x'],color='#5c5a5a', linewidth=0.6)
ax_scaled.set_title("Normalized sample processed lead V6 image",{'font': 'serif'})
apply_custom_style(ax_scaled)
plt.show()
#scaled_data to CSV
normalized_scaled_df['x'].to_csv('scaled_lead_V6_sample.csv',index=False)
#reading CSV to verify 1D and transformation
lead_V6_sample=pd.read_csv('scaled_lead_V6_sample.csv')
print(lead_V6_sample.shape)
lead_V6_sample
(255, 1)
| x | |
|---|---|
| 0 | 0.482524 |
| 1 | 0.527031 |
| 2 | 0.571109 |
| 3 | 0.559550 |
| 4 | 0.580587 |
| ... | ... |
| 250 | 0.764597 |
| 251 | 0.706526 |
| 252 | 0.685789 |
| 253 | 0.646040 |
| 254 | 0.591835 |
255 rows × 1 columns
# Plotting the 1D signal
lead_V6_sample = pd.DataFrame(lead_V6_sample, columns = ['x'])
fig_1d_signal, ax_1d_signal = plt.subplots()
plt.gca().invert_yaxis()
ax_1d_signal.plot(lead_V6_sample,color='#5c5a5a', linewidth=0.6)
ax_1d_signal.set_title("Normalized 1D sample processed lead V6 image",{'font': 'serif'})
apply_custom_style(ax_1d_signal)
plt.show()
